Skip to content

[moe training] refactor to share benchmarking and profiling utils #2767

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 15, 2025
Merged

[moe training] refactor to share benchmarking and profiling utils #2767

merged 1 commit into from
Aug 15, 2025

Conversation

danielvegamyhre
Copy link
Contributor

@danielvegamyhre danielvegamyhre commented Aug 14, 2025 •
edited
Loading

Stacked PRs:

[moe training] refactor to share benchmarking and profiling utils

MoE layer FSDP fwd + bwd bench

rm -rf /tmp/torchinductor_danvm; torchrun --nproc-per-node=8 --local-ranks-filter=0 benchmarks/prototype/moe_training/benchmark_moe_fsdp.py --compile  --recipe="fp8_rowwwise"

...
BF16 time: 100860.36600000001 us
Scaled time: 110353.22200000001 us
Speedup: 0.914x

Scaled grouped mm with dynamic quant fwd + bwd bench

rm -rf /tmp/torchinductor_danvm; torchrun --nproc-per-node=8 --local-ranks-filter=0 benchmarks/prototype/moe_training/benchmark_scaled_grouped_mm_dq.py --compile 

...

A_shape        B_shape             bf16_time_us    scaled_time_us  fp8_speedup
-------------  ----------------  --------------  ----------------  -------------
(16640, 5120)  (16, 8192, 5120)         10523.7           12889.3  0.816x

@danielvegamyhre
[moe training] refactor to share benchmarking and profiling utils
89b9868 
stack-info: PR: #2767, branch: danielvegamyhre/stack/41
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2767

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 14, 2025
danielvegamyhre added a commit that referenced this pull request Aug 14, 2025
@danielvegamyhre
[moe training] refactor to share benchmarking and profiling utils
01ade5c 
stack-info: PR: #2767, branch: danielvegamyhre/stack/41
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/41 branch from 3401bb4 to 01ade5c Compare August 14, 2025 00:03
This was referenced Aug 14, 2025
Merged
Merged
Merged
Merged
Merged
Merged
@danielvegamyhre danielvegamyhre changed the base branch from danielvegamyhre/stack/40 to main August 14, 2025 00:07
danielvegamyhre added a commit that referenced this pull request Aug 14, 2025
@danielvegamyhre
[moe training] refactor to share benchmarking and profiling utils
c02d02b 
stack-info: PR: #2767, branch: danielvegamyhre/stack/41
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/41 branch from 01ade5c to c02d02b Compare August 14, 2025 00:07
@danielvegamyhre danielvegamyhre changed the base branch from main to danielvegamyhre/stack/40 August 14, 2025 00:08
@danielvegamyhre danielvegamyhre added the topic: not user facing Use this tag if you don't want this PR to show up in release notes label Aug 14, 2025
@danielvegamyhre danielvegamyhre changed the base branch from danielvegamyhre/stack/40 to main August 14, 2025 01:04
danielvegamyhre added a commit that referenced this pull request Aug 14, 2025
@danielvegamyhre
[moe training] refactor to share benchmarking and profiling utils
43ffe7a 
stack-info: PR: #2767, branch: danielvegamyhre/stack/41
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/41 branch from c02d02b to 43ffe7a Compare August 14, 2025 01:04
@danielvegamyhre danielvegamyhre changed the base branch from main to danielvegamyhre/stack/40 August 14, 2025 01:05
@danielvegamyhre danielvegamyhre mentioned this pull request Aug 14, 2025
Draft
@danielvegamyhre danielvegamyhre changed the base branch from danielvegamyhre/stack/40 to main August 14, 2025 01:44
@danielvegamyhre danielvegamyhre changed the base branch from main to danielvegamyhre/stack/40 August 14, 2025 01:45
@danielvegamyhre danielvegamyhre mentioned this pull request Aug 14, 2025
Merged
@danielvegamyhre danielvegamyhre changed the base branch from danielvegamyhre/stack/40 to main August 14, 2025 05:51
@danielvegamyhre danielvegamyhre changed the base branch from main to danielvegamyhre/stack/40 August 14, 2025 05:51
@danielvegamyhre danielvegamyhre changed the base branch from danielvegamyhre/stack/40 to main August 14, 2025 21:27
@danielvegamyhre danielvegamyhre changed the base branch from main to danielvegamyhre/stack/40 August 14, 2025 21:28
@danielvegamyhre danielvegamyhre changed the base branch from danielvegamyhre/stack/40 to main August 14, 2025 22:23
@danielvegamyhre danielvegamyhre changed the base branch from main to danielvegamyhre/stack/40 August 14, 2025 22:23
@danielvegamyhre danielvegamyhre changed the base branch from danielvegamyhre/stack/40 to main August 14, 2025 22:34
@danielvegamyhre danielvegamyhre changed the base branch from main to danielvegamyhre/stack/40 August 14, 2025 22:34
@danielvegamyhre danielvegamyhre changed the base branch from danielvegamyhre/stack/40 to main August 14, 2025 23:20
@danielvegamyhre danielvegamyhre changed the base branch from main to danielvegamyhre/stack/40 August 14, 2025 23:20
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/40 branch from 23f4c81 to 033a808 Compare August 15, 2025 16:21
danielvegamyhre added a commit that referenced this pull request Aug 15, 2025
@danielvegamyhre
[moe training] refactor to share benchmarking and profiling utils
24cda99 
stack-info: PR: #2767, branch: danielvegamyhre/stack/41
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/41 branch from 43ffe7a to 24cda99 Compare August 15, 2025 16:21
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/40 branch from 033a808 to c95b3ee Compare August 15, 2025 16:22
danielvegamyhre added a commit that referenced this pull request Aug 15, 2025
@danielvegamyhre
[moe training] refactor to share benchmarking and profiling utils
52ec710 
stack-info: PR: #2767, branch: danielvegamyhre/stack/41
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/41 branch from 24cda99 to 52ec710 Compare August 15, 2025 16:22
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/40 branch from c95b3ee to a1ffe86 Compare August 15, 2025 16:23
danielvegamyhre added a commit that referenced this pull request Aug 15, 2025
@danielvegamyhre
[moe training] refactor to share benchmarking and profiling utils
bb57600 
stack-info: PR: #2767, branch: danielvegamyhre/stack/41
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/41 branch from 52ec710 to bb57600 Compare August 15, 2025 16:23
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/40 branch from a1ffe86 to 65e98ad Compare August 15, 2025 16:26
danielvegamyhre added a commit that referenced this pull request Aug 15, 2025
@danielvegamyhre
[moe training] refactor to share benchmarking and profiling utils
db9cdc9 
stack-info: PR: #2767, branch: danielvegamyhre/stack/41
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/41 branch from bb57600 to db9cdc9 Compare August 15, 2025 16:26
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/40 branch from 65e98ad to 6afefbc Compare August 15, 2025 16:26
danielvegamyhre added a commit that referenced this pull request Aug 15, 2025
@danielvegamyhre
[moe training] refactor to share benchmarking and profiling utils
c8c7743 
stack-info: PR: #2767, branch: danielvegamyhre/stack/41
@danielvegamyhre danielvegamyhre force-pushed the danielvegamyhre/stack/41 branch 2 times, most recently from c8c7743 to 89b9868 Compare August 15, 2025 16:28
@danielvegamyhre danielvegamyhre changed the base branch from danielvegamyhre/stack/40 to main August 15, 2025 16:28
@danielvegamyhre danielvegamyhre merged commit aed4f84 into main Aug 15, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vkuzo vkuzo vkuzo approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. topic: not user facing Use this tag if you don't want this PR to show up in release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@danielvegamyhre @vkuzo